NIST Workshop on Ontology Evaluation

The National Institute for Standards and Technology sponsored a workshop in October, 2007, on the subject of ontology evaluation. An international group of invited experts met for two days to discuss problems in measuring ontology quality. The workshop highlighted several divisions among ontology developers regarding approaches to ontology evaluation. These divisions were generally reflective of the opinions of the participants. However, the workshop documented a paucity of empirical evidence in support of any particular position. Given the importance of ontologies to every knowledge-intensive human activity, there is an urgent need for research to develop an empirically derived knowledge base of best practices in ontology engineering and methods for assuring ontology quality over time. This is a report of the workshop discussion and brainstorming by the participants about what such a research program might look like.